Skip to content

test(test-suite): add e2e block cap tests for HCU metering#2046

Closed
Eikix wants to merge 32 commits intomainfrom
host-statevar-tests
Closed

test(test-suite): add e2e block cap tests for HCU metering#2046
Eikix wants to merge 32 commits intomainfrom
host-statevar-tests

Conversation

@Eikix
Copy link
Copy Markdown
Contributor

@Eikix Eikix commented Mar 5, 2026

Summary

  • Add 5 block-cap test scenarios to test-suite/e2e/test/encryptedERC20/EncryptedERC20.HCU.ts exercising HCULimit through real EncryptedERC20 FHE operations on the deployed stack
  • Wire into CI pipeline via fhevm-cli test hcu-block-cap and a new workflow step in test-suite-e2e-tests.yml
  • Add HCU_LIMIT_CONTRACT_ADDRESS and DEPLOYER_PRIVATE_KEY env vars to staging config

Test scenarios

Scenario What it validates
Multi-user accumulation Two users' FHE transfers in the same block sum correctly in the block meter
Cap exhaustion Second tx reverts when block HCU cap is exceeded
Block rollover Meter resets after a new block; previously blocked caller succeeds
Whitelist removal Whitelisted contract skips metering; after removal, metering resumes
Non-owner rejection setHCUPerBlock reverts when called by non-owner

Closes zama-ai/fhevm-internal#1099

Test plan

  • Trigger fhevm-cli test hcu-block-cap on a deployed local stack (see below)
  • Run the full E2E workflow via workflow_dispatch on this branch
  • Verify all 5 new tests pass and existing tests are unaffected

Local testing strategy

# 1. Deploy the full fhevm stack
cd test-suite/fhevm
./fhevm-cli deploy

# 2. Run only the new block cap tests
./fhevm-cli test hcu-block-cap

# 3. Run the existing HCU + ERC20 tests to check for regressions
./fhevm-cli test erc20

# 4. Cleanup
./fhevm-cli clean

Add 5 block-cap scenarios to the E2E test suite exercising HCULimit
through real EncryptedERC20 FHE operations on the deployed stack:
multi-user accumulation, cap exhaustion, block rollover, whitelist
removal, and non-owner rejection.

Wire into CI via `fhevm-cli test hcu-block-cap` and a new workflow step.
@Eikix Eikix requested a review from a team as a code owner March 5, 2026 10:28
@cla-bot cla-bot bot added the cla-signed label Mar 5, 2026
@mergify
Copy link
Copy Markdown

mergify bot commented Mar 5, 2026

🧪 CI Insights

Here's what we observed from your CI run for 1d11f82.

🟢 All jobs passed!

But CI Insights is watching 👀

Eikix added 2 commits March 5, 2026 11:56
- Rework block rollover test to actually block a caller in block N,
  then verify that same caller succeeds after rollover in block N+1
- Add missing DEPLOYER_PRIVATE_KEY to .env.example
- Accumulation test: use greaterThan instead of exact equality
  (block meter vs receipt HCU have a small discrepancy on real infra)
- Cap exhaustion + rollover tests: pass explicit gasLimit to bypass
  estimateGas, which reverts against pending state when cap is filled
@Eikix
Copy link
Copy Markdown
Contributor Author

Eikix commented Mar 5, 2026

Local E2E Test Results

All 5 block cap tests pass on a locally deployed stack:

  EncryptedERC20:HCU
    block cap scenarios
      ✔ should accumulate HCU from multiple users in the same block (24633ms)
      ✔ should revert when block HCU cap is exhausted (24820ms)
      ✔ should allow previously blocked caller to succeed after block rollover (33315ms)
      ✔ should count HCU after whitelist removal (16760ms)
      ✔ should reject setHCUPerBlock from non-owner

  5 passing (2m)

ERC20 regression test also passes.

Issues found and fixed during local testing

  1. Docker Desktop buildx race — multiple compose targets sharing the same image tag fail with image already exists. Workaround: COMPOSE_BAKE=false (not needed on Linux CI).

  2. Block meter vs receipt HCU discrepancy — on real infra, the block meter (511) didn't exactly match the sum of receipt HCUs (519). Relaxed accumulation assertion to greaterThan(max(hcuAlice, hcuBob)) instead of exact equality.

  3. estimateGas reverts against pending state — when automine is off and the first tx fills the block cap, submitting the second tx fails at gas estimation (not at mining). Fixed by passing explicit gasLimit: 1_000_000 to bypass estimation for txs expected to revert.

Eikix added 10 commits March 5, 2026 12:20
Replace loose greaterThan check with near-sum assertion allowing ~2%
drift between receipt-reported HCU and on-chain block meter.
…ion assertion

The receipt parser reconstructs HCU from the @fhevm/solidity npm price
table while the block meter uses the deployed contract's hardcoded
prices. A version skew between the two causes a small discrepancy.
Instead of cross-comparing with tolerance, assert the block meter
exceeds each individual tx's HCU — proving accumulation without
depending on price table parity.
Add NotHostOwner error to HCU_LIMIT_ABI and assert the specific custom
error instead of generic revert.
- Scope save/restore of HCU limits to only the 2 tests that lower them
  (nested describe with its own beforeEach/afterEach)
- Extract mintAndDistribute helper for repeated mint+transfer preamble
- Remove blanket whitelist cleanup from afterEach (test cleans up itself)
- Parallelize 3 sequential view calls with Promise.all
Replace receipt-based HCU comparison with three block meter readings:
1. Single-tx block → baseline meter
2. Two-tx block → meter exceeds baseline (proves accumulation)
3. Single-tx block → meter resets and matches baseline

No cross-comparison of price tables, no getTxHCUFromTxReceipt needed.
- Assert meter2 == 2 * meter1 (exact, same ops in both txs)
- Remove unnecessary mineNBlocks between blocks (meter resets
  automatically in each new block)
DO NOT MERGE — revert before merge. Added `if: false` to all test
steps except HCU block cap to validate in isolation.
The CI was pulling the pre-built test-suite Docker image (v0.11.0-1)
which doesn't contain the new block cap scenarios tests. Use --build
so the image is built from the current checkout.
…rtion

- NotHostOwner takes an address parameter: error NotHostOwner(address)
- Relax meter2 == meter1*2 to meter2 > meter1 since alice→bob and
  bob→alice can differ slightly in HCU due to balance init paths
@Eikix Eikix marked this pull request as draft March 5, 2026 17:35
Eikix added 7 commits March 5, 2026 19:02
The same alice→bob transfer produces slightly different HCU across
runs due to balance state changes from intermediate transfers.
Assert reset behavior (meter3 > 0 and meter3 < meter2) instead of
exact equality with meter1.
Anvil runs with --block-time 1, so blocks keep getting mined even
with evm_setAutomine(false). Use evm_setIntervalMining(0) to fully
pause block production, then restore both after mining.
…rEach

Disable interval mining once in beforeEach (deterministic blocks),
restore in afterEach. Tests only toggle automine for batching.
Disabling interval mining in beforeEach hangs because Anvil's
evm_setIntervalMining(0) overrides automine. Revert to the per-test
pattern (disable interval+automine before batching, restore after)
which passed in CI run 22733231829.
Restore workflow to match main, keeping only the new HCU block cap
test step addition.
@Eikix
Copy link
Copy Markdown
Contributor Author

Eikix commented Mar 6, 2026

Added E2E HCU block-cap coverage for accumulation, cap exhaustion, rollover, whitelist removal, and non-owner rejection.

Also hardened test cleanup so HCU limits, whitelist state, and mining settings are restored after each case, which avoids leaking state into later tests.

Local validation:

  • ./fhevm-cli test hcu-block-cap passed
  • ./fhevm-cli test erc20 passed immediately after on the same stack

@Eikix
Copy link
Copy Markdown
Contributor Author

Eikix commented Mar 6, 2026

@claude /pr-review

@Eikix Eikix marked this pull request as ready for review March 6, 2026 09:42
@zama-ai zama-ai deleted a comment from claude bot Mar 6, 2026
mergify bot added a commit that referenced this pull request Mar 6, 2026
Eikix added 9 commits March 9, 2026 11:06
The automine=true + intervalMining=0 combo is unreliable in CI —
Anvil hangs for ~5min before mining the mint tx, causing Mocha timeout.

Switch to automine=false + explicit evm_mine after each tx, matching
the proven pattern used by the "with lowered limits" tests that pass
consistently. Also add gasLimit overrides to bypass estimateGas against
pending state.
…ploy

Forward --resume STEP and --only STEP flags from fhevm-cli to the
underlying deploy-fhevm-stack.sh script, with step validation and
mutual exclusivity check.

Use --only test-suite in CI when deploy-build is set, so only the
test-suite image is rebuilt from the branch instead of the entire stack.
cleanup_single_step and cleanup_from_step used --remove-orphans with
a single compose file, causing Docker Compose to tear down every
container in the project not defined in that file. This destroyed the
entire stack when running e.g. --only test-suite.
The --only test-suite approach rebuilds only the test container but
uses pre-built host-sc images that lack the HCULimit contract. The
HCU block cap tests need host-sc built from the branch, so we must
use the full --build deploy for now.

The --resume/--only CLI flags and the --remove-orphans fix in the
deploy script are kept — they're useful for local development and
future CI optimizations.
Keep the PR scoped to the HCU whitelist test fix and the deploy-build
workflow input. The --resume/--only CLI flags and --remove-orphans fix
can be submitted in a separate PR.
@Eikix
Copy link
Copy Markdown
Contributor Author

Eikix commented Mar 10, 2026

@Mergifyio queue

@mergify
Copy link
Copy Markdown

mergify bot commented Mar 10, 2026

Merge Queue Status

  • 🟠 Waiting for queue conditions
  • ⏳ Enter queue
  • ⏳ Run checks
  • ⏳ Merge
Required conditions to enter a queue
  • -closed [📌 queue requirement]
  • any of [🔀 queue conditions]:
    • all of [📌 queue conditions of queue main]:
      • #review-threads-unresolved = 0 [🛡 GitHub branch protection]
      • any of [🛡 GitHub branch protection]:
        • check-neutral = common-pull-request/lint (bpr)
        • check-skipped = common-pull-request/lint (bpr)
        • check-success = common-pull-request/lint (bpr)
      • #approved-reviews-by >= 1 [🛡 GitHub branch protection]
      • #changes-requested-reviews-by = 0 [🛡 GitHub branch protection]
      • base = main
      • branch-protection-review-decision = APPROVED [🛡 GitHub branch protection]
      • label!=do-not-merge
      • any of [🛡 GitHub branch protection]:
        • check-skipped = coprocessor-cargo-listener-tests/cargo-tests (bpr)
        • check-neutral = coprocessor-cargo-listener-tests/cargo-tests (bpr)
        • check-success = coprocessor-cargo-listener-tests/cargo-tests (bpr)
      • any of [🛡 GitHub branch protection]:
        • check-skipped = coprocessor-cargo-test/cargo-tests (bpr)
        • check-neutral = coprocessor-cargo-test/cargo-tests (bpr)
        • check-success = coprocessor-cargo-test/cargo-tests (bpr)
      • any of [🛡 GitHub branch protection]:
        • check-skipped = coprocessor-dependency-analysis/dependencies-check (bpr)
        • check-neutral = coprocessor-dependency-analysis/dependencies-check (bpr)
        • check-success = coprocessor-dependency-analysis/dependencies-check (bpr)
      • any of [🛡 GitHub branch protection]:
        • check-skipped = gateway-contracts-deployment-tests/sc-deploy (bpr)
        • check-neutral = gateway-contracts-deployment-tests/sc-deploy (bpr)
        • check-success = gateway-contracts-deployment-tests/sc-deploy (bpr)
      • any of [🛡 GitHub branch protection]:
        • check-skipped = kms-connector-tests/test-connector (bpr)
        • check-neutral = kms-connector-tests/test-connector (bpr)
        • check-success = kms-connector-tests/test-connector (bpr)
  • -conflict [📌 queue requirement]
  • -draft [📌 queue requirement]
  • any of [📌 queue -> configuration change requirements]:
    • -mergify-configuration-changed
    • check-success = Configuration changed

@Eikix
Copy link
Copy Markdown
Contributor Author

Eikix commented Mar 10, 2026

@Mergifyio refresh

@mergify
Copy link
Copy Markdown

mergify bot commented Mar 10, 2026

refresh

✅ Pull request refreshed

@Eikix
Copy link
Copy Markdown
Contributor Author

Eikix commented Mar 10, 2026

@Mergifyio queue

@mergify
Copy link
Copy Markdown

mergify bot commented Mar 10, 2026

queue

☑️ Command queue ignored because it is already running from a previous command.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants